Exploring the Role of Infodemics in People's Incompliance with Preventive Measures during the COVID-19 in Conflict Settings (Mixed Method Study).

The evolving availability of health information on social media, regardless of its credibility, raises several questions about its impact on our health decisions and social behaviors, especially during health crises and in conflict settings where compliance with preventive measures and health guidelines is already a challenge due to socioeconomic factors. For these reasons, we assessed compliance with preventive measures and investigated the role of infodemic in people's non-compliance with COVID-19 containment measures in Yemen. To this purpose and to triangulate our data collection, we executed a mixed method approach in which raw aggregated data were taken and analyzed from multiple sources (COVID-19 Government Response Tracker and Google COVID-19 Community Mobility Reports), then complemented and verified with In-depth interviews. Our results showed that the population in Yemen had relatively complied with the governmental containment measures at the beginning of the pandemic. However, containment measures were not supported by daily COVID-19 reports due to low transparency, which, together with misinformation and lack of access to reliable sources, has caused the population not to believe in COVID-19 and even practice social pressure on those who showed some compliance with the WHO guidelines. Those results indicate the importance of adopting an infodemic management approach in response to future outbreaks, particularly in conflict settings.

Measuring the Burden of Infodemics: Summary of the Methods and Results of the Fifth WHO Infodemic Management Conference.

Background: An infodemic is excess information, including false or misleading information, that spreads in digital and physical environments during a public health emergency. The COVID-19 pandemic has been accompanied by an unprecedented global infodemic that has led to confusion about the benefits of medical and public health interventions, with substantial impact on risk-taking and health-seeking behaviors, eroding trust in health authorities and compromising the effectiveness of public health responses and policies. Standardized measures are needed to quantify the harmful impacts of the infodemic in a systematic and methodologically robust manner, as well as harmonizing highly divergent approaches currently explored for this purpose. This can serve as a foundation for a systematic, evidence-based approach to monitoring, identifying, and mitigating future infodemic harms in emergency preparedness and prevention. Objective: In this paper, we summarize the Fifth World Health Organization (WHO) Infodemic Management Conference structure, proceedings, outcomes, and proposed actions seeking to identify the interdisciplinary approaches and frameworks needed to enable the measurement of the burden of infodemics. Methods: An iterative human-centered design (HCD) approach and concept mapping were used to facilitate focused discussions and allow for the generation of actionable outcomes and recommendations. The discussions included 86 participants representing diverse scientific disciplines and health authorities from 28 countries across all WHO regions, along with observers from civil society and global public health-implementing partners. A thematic map capturing the concepts matching the key contributing factors to the public health burden of infodemics was used throughout the conference to frame and contextualize discussions. Five key areas for immediate action were identified. Results: The 5 key areas for the development of metrics to assess the burden of infodemics and associated interventions included (1) developing standardized definitions and ensuring the adoption thereof; (2) improving the map of concepts influencing the burden of infodemics; (3) conducting a review of evidence, tools, and data sources; (4) setting up a technical working group; and (5) addressing immediate priorities for postpandemic recovery and resilience building. The summary report consolidated group input toward a common vocabulary with standardized terms, concepts, study designs, measures, and tools to estimate the burden of infodemics and the effectiveness of infodemic management interventions. Conclusions: Standardizing measurement is the basis for documenting the burden of infodemics on health systems and population health during emergencies. Investment is needed into the development of practical, affordable, evidence-based, and systematic methods that are legally and ethically balanced for monitoring infodemics; generating diagnostics, infodemic insights, and recommendations; and developing interventions, action-oriented guidance, policies, support options, mechanisms, and tools for infodemic managers and emergency program managers.

COVID-19 Misinformation Detection: Machine-Learned Solutions to the Infodemic.

Background: The volume of COVID-19-related misinformation has long exceeded the resources available to fact checkers to effectively mitigate its ill effects. Automated and web-based approaches can provide effective deterrents to online misinformation. Machine learning-based methods have achieved robust performance on text classification tasks, including potentially low-quality-news credibility assessment. Despite the progress of initial, rapid interventions, the enormity of COVID-19-related misinformation continues to overwhelm fact checkers. Therefore, improvement in automated and machine-learned methods for an infodemic response is urgently needed. Objective: The aim of this study was to achieve improvement in automated and machine-learned methods for an infodemic response. Methods: We evaluated three strategies for training a machine-learning model to determine the highest model performance: (1) COVID-19-related fact-checked data only, (2) general fact-checked data only, and (3) combined COVID-19 and general fact-checked data. We created two COVID-19-related misinformation data sets from fact-checked "false" content combined with programmatically retrieved "true" content. The first set contained ~7000 entries from July to August 2020, and the second contained ~31,000 entries from January 2020 to June 2022. We crowdsourced 31,441 votes to human label the first data set. Results: The models achieved an accuracy of 96.55% and 94.56% on the first and second external validation data set, respectively. Our best-performing model was developed using COVID-19-specific content. We were able to successfully develop combined models that outperformed human votes of misinformation. Specifically, when we blended our model predictions with human votes, the highest accuracy we achieved on the first external validation data set was 99.1%. When we considered outputs where the machine-learning model agreed with human votes, we achieved accuracies up to 98.59% on the first validation data set. This outperformed human votes alone with an accuracy of only 73%. Conclusions: External validation accuracies of 96.55% and 94.56% are evidence that machine learning can produce superior results for the difficult task of classifying the veracity of COVID-19 content. Pretrained language models performed best when fine-tuned on a topic-specific data set, while other models achieved their best accuracy when fine-tuned on a combination of topic-specific and general-topic data sets. Crucially, our study found that blended models, trained/fine-tuned on general-topic content with crowdsourced data, improved our models' accuracies up to 99.7%. The successful use of crowdsourced data can increase the accuracy of models in situations when expert-labeled data are scarce. The 98.59% accuracy on a "high-confidence" subsection comprised of machine-learned and human labels suggests that crowdsourced votes can optimize machine-learned labels to improve accuracy above human-only levels. These results support the utility of supervised machine learning to deter and combat future health-related disinformation.

COVID-19 and Public Health: Analysis of Opinions in Social Media.

The article presents the results of research of public opinion during the third wave of the COVID-19 pandemic in Russia. The study touches on the attitude of citizens to public health, as well as the reaction of social media users to government measures in a crisis situation during a pandemic. Special attention is paid to the phenomenon of infodemic and methods of detecting cases of the spread of false and unverified information about diseases. The article demonstrates the application of an interdisciplinary approach using network analysis of texts and sociological research. A model for detecting social stress in the textual communication of social network users using a specially trained neural network and linguistic analysis methods is presented. The validity and validity of the results of the analysis of social network data were verified using a sociological survey. This approach allows us to identify points of tension in matters of public health promotion, during crisis situations to improve interaction between the government and society, and to timely adjust government plans and actions to ensure resilience in emergency situations for public health purposes.

Infodemic Management Using Digital Information and Knowledge Cocreation to Address COVID-19 Vaccine Hesitancy: Case Study From Ghana.

Background: Infodemic management is an integral part of pandemic management. Ghana Health Services (GHS) together with the UNICEF (United Nations International Children's Emergency Fund) Country Office have developed a systematic process that effectively identifies, analyzes, and responds to COVID-19 and vaccine-related misinformation in Ghana. Objective: This paper describes an infodemic management system workflow based on digital data collection, qualitative methodology, and human-centered systems to support the COVID-19 vaccine rollout in Ghana with examples of system implementation. Methods: The infodemic management system was developed by the Health Promotion Division of the GHS and the UNICEF Country Office. It uses Talkwalker, a social listening software platform, to collect misinformation on the web. The methodology relies on qualitative data analysis and interpretation as well as knowledge cocreation to verify the findings. Results: A multi-sectoral National Misinformation Task Force was established to implement and oversee the misinformation management system. Two members of the task force were responsible for carrying out the analysis. They used Talkwalker to find posts that include the keywords related to COVID-19 vaccine-related discussions. They then assessed the significance of the posts on the basis of the engagement rate and potential reach of the posts, negative sentiments, and contextual factors. The process continues by identifying misinformation within the posts, rating the risk of identified misinformation posts, and developing proposed responses to address them. The results of the analysis are shared weekly with the Misinformation Task Force for their review and verification to ensure that the risk assessment and responses are feasible, practical, and acceptable in the context of Ghana. Conclusions: The paper describes an infodemic management system workflow in Ghana based on qualitative data synthesis that can be used to manage real-time infodemic responses.

Digital Visual Communication for Public Health: Design Proposal for a Vaccinated Emoji.

In the 21st century, the internet and particularly social media have become essential platforms for the spread of health information (including misinformation and disinformation). One of the distinguishing features of communication on these platforms is the widespread use of emojis. Though seemingly trivial emojis are now used by many if not most public health figures and organizations alongside important health updates. Much of that information has had to do with vaccination. Vaccines are a critical public health tool but one surrounded by falsehoods, phobias, and misinformation fueling vaccine hesitancy. Part of that has to do with their lack of positive representation on social media (eg, the syringe emoji is a plain needle, which for many people is an uncomfortable image). We thus argue that vaccination deserves an entirely new emoji to communicate vaccine confidence and discuss a design proposal for a vaccinated emoji that has gained traction in the global public health community.

Social science approaches to infodemiology: understanding the social, political, and economic context of information.

Embedded within the COVID-19 pandemic is the spread of a new pandemic of information - some accurate, some not - that can challenge the public health response. This has been termed an 'infodemic' and infodemic management is now a major feature of the World Health Organization's work on health emergencies. This commentary highlights political, social, and economic aspects of infodemics and posits social science as critical to mitigating the current infodemic and preventing future ones. Infodemic managers should address the wider context of infodemics if we are to understand narratives, help to craft positive ones, and confront the root causes of misinformation rather than just the symptoms.

A Public Health Research Agenda for Managing Infodemics: Methods and Results of the First WHO Infodemiology Conference.

BACKGROUND: An infodemic is an overflow of information of varying quality that surges across digital and physical environments during an acute public health event. It leads to confusion, risk-taking, and behaviors that can harm health and lead to erosion of trust in health authorities and public health responses. Owing to the global scale and high stakes of the health emergency, responding to the infodemic related to the pandemic is particularly urgent. Building on diverse research disciplines and expanding the discipline of infodemiology, more evidence-based interventions are needed to design infodemic management interventions and tools and implement them by health emergency responders. OBJECTIVE: The World Health Organization organized the first global infodemiology conference, entirely online, during June and July 2020, with a follow-up process from August to October 2020, to review current multidisciplinary evidence, interventions, and practices that can be applied to the COVID-19 infodemic response. This resulted in the creation of a public health research agenda for managing infodemics. METHODS: As part of the conference, a structured expert judgment synthesis method was used to formulate a public health research agenda. A total of 110 participants represented diverse scientific disciplines from over 35 countries and global public health implementing partners. The conference used a laddered discussion sprint methodology by rotating participant teams, and a managed follow-up process was used to assemble a research agenda based on the discussion and structured expert feedback. This resulted in a five-workstream frame of the research agenda for infodemic management and 166 suggested research questions. The participants then ranked the questions for feasibility and expected public health impact. The expert consensus was summarized in a public health research agenda that included a list of priority research questions. RESULTS: The public health research agenda for infodemic management has five workstreams: (1) measuring and continuously monitoring the impact of infodemics during health emergencies; (2) detecting signals and understanding the spread and risk of infodemics; (3) responding and deploying interventions that mitigate and protect against infodemics and their harmful effects; (4) evaluating infodemic interventions and strengthening the resilience of individuals and communities to infodemics; and (5) promoting the development, adaptation, and application of interventions and toolkits for infodemic management. Each workstream identifies research questions and highlights 49 high priority research questions. CONCLUSIONS: Public health authorities need to develop, validate, implement, and adapt tools and interventions for managing infodemics in acute public health events in ways that are appropriate for their countries and contexts. Infodemiology provides a scientific foundation to make this possible. This research agenda proposes a structured framework for targeted investment for the scientific community, policy makers, implementing organizations, and other stakeholders to consider.

Infodemic Signal Detection During the COVID-19 Pandemic: Development of a Methodology for Identifying Potential Information Voids in Online Conversations.

BACKGROUND: The COVID-19 pandemic has been accompanied by an infodemic: excess information, including false or misleading information, in digital and physical environments during an acute public health event. This infodemic is leading to confusion and risk-taking behaviors that can be harmful to health, as well as to mistrust in health authorities and public health responses. The World Health Organization (WHO) is working to develop tools to provide an evidence-based response to the infodemic, enabling prioritization of health response activities. OBJECTIVE: In this work, we aimed to develop a practical, structured approach to identify narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, thus providing actionable data to help the WHO prioritize its response efforts to address the COVID-19 infodemic. METHODS: We developed a taxonomy to filter global public conversations in English and French related to COVID-19 on social media into 5 categories with 35 subcategories. The taxonomy and its implementation were validated for retrieval precision and recall, and they were reviewed and adapted as language about the pandemic in online conversations changed over time. The aggregated data for each subcategory were analyzed on a weekly basis by volume, velocity, and presence of questions to detect signals of information voids with potential for confusion or where mis- or disinformation may thrive. A human analyst reviewed and identified potential information voids and sources of confusion, and quantitative data were used to provide insights on emerging narratives, influencers, and public reactions to COVID-19-related topics. RESULTS: A COVID-19 public health social listening taxonomy was developed, validated, and applied to filter relevant content for more focused analysis. A weekly analysis of public online conversations since March 23, 2020, enabled quantification of shifting interests in public health-related topics concerning the pandemic, and the analysis demonstrated recurring voids of verified health information. This approach therefore focuses on the detection of infodemic signals to generate actionable insights to rapidly inform decision-making for a more targeted and adaptive response, including risk communication. CONCLUSIONS: This approach has been successfully applied to identify and analyze infodemic signals, particularly information voids, to inform the COVID-19 pandemic response. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently recur and narratives shift over time. The approach is being piloted in individual countries and WHO regions to generate localized insights and actions; meanwhile, a pilot of an artificial intelligence-based social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Beyond the COVID-19 pandemic, the taxonomy and methodology may be adapted for fast deployment in future public health events, and they could form the basis of a routine social listening program for health preparedness and response planning.

EARS - A WHO Platform for AI-Supported Real-Time Online Social Listening of COVID-19 Conversations.

As the COVID-19 pandemic evolves, the accompanying infodemic is being amplified through social media and has challenged effective response. The WHO Early AI-supported Response with Social Listening (EARS) is a platform that summarizes real-time information about how people are talking about COVID-19 in public spaces online in 20 pilot countries and in four languages. The aim of the platform is to better integrate social listening with other data sources and analyses that can inform infodemic response.

WHO Digital Intelligence Analysis for Tracking Narratives and Information Voids in the COVID-19 Infodemic.

The COVID-19 pandemic is the first to unfold in the highly digitalized society of the 21st century and is therefore the first pandemic to benefit from and be threatened by a thriving real-time digital information ecosystem. For this reason, the response to the infodemic required development of a public health social listening taxonomy, a structure that can simplify the chaotic information ecosystem to enable an adaptable monitoring infrastructure that detects signals of fertile ground for misinformation and guides trusted sources of verified information to fill in information voids in a timely manner. A weekly analysis of public online conversations since 23 March 2020 has enabled the quantification of running shifts of public interest in public health-related topics concerning the pandemic and has demonstrated the frequent resumption of information voids relevant for public health interventions and risk communication in an emergency response setting.